QoS-Aware Data Replication in Hadoop Distributed File System
ثبت نشده
چکیده
Dr. Sunita Varma Department of ComputerTechnology and Application S. G. S. I. T. S. Indore, (M. P.), India [email protected] Ms. Gopi Khatri Department of Computer Engineering S. G. S. I. T. S Indore, (M. P.), India [email protected] --------------------------------------------------------------------ABSTRACT------------------------------------------------------------Cloud computing provides services using virtualized resources through Internet on pay per use basis. These services are delivered from millions of data centers which are connected with each other. Cloud system consists of commodity machines. The client data is stored on these machines. Probability of hardware failure and data corruption of these low performance machines are high. For fault tolerance and improving the reliability of the cloud system the data is replicated to multiple systems. Hadoop Distributed File System (HDFS) is used for distributed storage in cloud system. The data is stored in the form of fixed-size blocks i.e. 64MB. The data stored in HDFS is replicated on multiple systems for improving the reliability of the cloud system. Block replica placement algorithm is used in HDFS for replicating the data block. In this algorithm, QoS parameter for replicating the data block is not specified between client and service provider in the form of service level agreement. In this paper, an algorithm QoS-Aware Data Replication in HDFS is suggested which considers the QoS parameter for replicating the data block. The QoS parameter considered is expected replication time of application. The block of data is replicated to remote rack DataNodes which satisfies replication time requirement of application. This algorithm reduces the replication cost as compared to existing algorithm thus, improving the reliability and performance of system. Keywords-Cloud computing; quality of service; data replication; Hadoop distributed file system; replication cost. ------------------------------------------------------------------------------------------------------------------------------------------------Date of Submission: July 09, 2015 Date of Acceptance: Aug 12, 2015 ------------------------------------------------------------------------------------------------------------------------------------------------
منابع مشابه
Adaptive Dynamic Data Placement Algorithm for Hadoop in Heterogeneous Environments
Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop d...
متن کاملAn Experimental Evaluation of Performance of A Hadoop Cluster on Replica Management
Hadoop is an open source implementation of the MapReduce Framework in the realm of distributed processing. A Hadoop cluster is a unique type of computational cluster designed for storing and analyzing large datasets across cluster of workstations. To handle massive scale data, Hadoop exploits the Hadoop Distributed File System termed as HDFS. The HDFS similar to most distributed file systems sh...
متن کاملEfficient Data Replication Scheme based on Hadoop Distributed File System
Hadoop distributed file system (HDFS) is designed to store huge data set reliably, has been widely used for processing massive-scale data in parallel. In HDFS, the data locality problem is one of critical problem that causes the performance decrement of a file system. To solve the data locality problem, we propose an efficient data replication scheme based on access count prediction in a Hadoop...
متن کاملDelay Scheduling Based Replication Scheme for Hadoop Distributed File System
The data generated and processed by modern computing systems burgeon rapidly. MapReduce is an important programming model for large scale data intensive applications. Hadoop is a popular open source implementation of MapReduce and Google File System (GFS). The scalability and fault-tolerance feature of Hadoop makes it as a standard for BigData processing. Hadoop uses Hadoop Distributed File Sys...
متن کاملE2DR: Energy Efficient Data Replication in Data Grid
Abstract— Data grids are an important branch of gird computing which provide mechanisms for the management of large volumes of distributed data. Energy efficiency has recently emerged as a hot topic in large distributed systems. The development of computing systems is traditionally focused on performance improvements driven by the demand of client's applications in scientific and business domai...
متن کامل